Atespace: per-tenant scoping for the actor lifecycle by HavenXia · Pull Request #280 · agent-substrate/substrate

Haven Xia (HavenXia) · 2026-06-19T05:08:51Z

This is part of solution for #21.

First incremental slice of the atespace design for actors — part of #21. An atespace is a mandatory tenant boundary that every actor belongs to. It's folded into the actor's identity and storage key (actor:<atespace>:<id>), so list/get/delete within a tenant is a cheap key-prefix operation, and actors in different atespaces can reuse the same id without colliding.

This PR adds atespace through the actor lifecycle end-to-end (proto → store → control API → kubectl-ate).

It doesn't touch DNS, snapshots, scheduling, or auth. Landing it on its own so the future changes are additive, and everything that isn't actor-CRUD is explicitly out of scope below.

This PR changes:

Proto — every actor RPC request carries an atespace; Actor carries it as part of its identity.
Store — actors are keyed actor:<atespace>:<id> - GetActor/DeleteActor/ListActors take an atespace. Listing is a scoped SCAN actor:<atespace>:*, or SCAN actor:* for all atespaces.
Control API — create / get / delete / suspend / pause / resume / update are all atespace-scoped. atespace is required and validated as a DNS-1123 label. The syncer's dead-worker recovery is atespace-aware by adding Worker.actor_atespace.
changes on kubectl-ate:
- --atespace / -a on every actor subcommand (create/get/delete/resume/suspend/pause/logs).
- -A / --all-atespaces to list across all tenants.
- Add an ATESPACE column in the actor table, the existing namespace column is renamed TEMPLATE NS to disambiguate it from the atespace.

Examples

Scope a listing to one tenant (-a is shorthand for --atespace):

Creation

$ kubectl ate create actor test -t ate-demo-counter/counter --atespace team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID     STATUS             ATEOM POD   ATEOM IP   VERSION
team-a     ate-demo-counter   counter    test   STATUS_SUSPENDED   <none>                 1

$ kubectl ate create actor test2 -t ate-demo-counter/counter --atespace team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID      STATUS             ATEOM POD   ATEOM IP   VERSION
team-a     ate-demo-counter   counter    test2   STATUS_SUSPENDED   <none>                 1

$ kubectl ate create actor test -t ate-demo-counter/counter --atespace team-b
ATESPACE   TEMPLATE NS        TEMPLATE   ID     STATUS             ATEOM POD   ATEOM IP   VERSION
team-b     ate-demo-counter   counter    test   STATUS_SUSPENDED   <none>                 1

Get by atespaces

$ kubectl ate get actors -A
ATESPACE   TEMPLATE NS        TEMPLATE   ID      STATUS             ATEOM POD   ATEOM IP   VERSION
team-a     ate-demo-counter   counter    test    STATUS_SUSPENDED   <none>                 1
team-a     ate-demo-counter   counter    test2   STATUS_SUSPENDED   <none>                 1
team-b     ate-demo-counter   counter    test    STATUS_SUSPENDED   <none>                 1

$ kubectl ate get actors -a team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID      STATUS             ATEOM POD   ATEOM IP   VERSION
team-a     ate-demo-counter   counter    test    STATUS_SUSPENDED   <none>                 1
team-a     ate-demo-counter   counter    test2   STATUS_SUSPENDED   <none>                 1

$ kubectl ate get actors -a team-b
ATESPACE   TEMPLATE NS        TEMPLATE   ID     STATUS             ATEOM POD   ATEOM IP   VERSION
team-b     ate-demo-counter   counter    test   STATUS_SUSPENDED   <none>                 1

Resume & Suspend

$ kubectl ate resume actor test -a team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID     STATUS           ATEOM POD                                              ATEOM IP     VERSION
team-a     ate-demo-counter   counter    test   STATUS_RUNNING   ate-demo-counter/counter-deployment-5d9b77d6fd-r846n   10.52.3.86   3

$ kubectl ate get actors -a team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID      STATUS             ATEOM POD                                              ATEOM IP     VERSION
team-a     ate-demo-counter   counter    test    STATUS_RUNNING     ate-demo-counter/counter-deployment-5d9b77d6fd-r846n   10.52.3.86   3
team-a     ate-demo-counter   counter    test2   STATUS_SUSPENDED   <none>                                                              1

$ kubectl ate suspend actor test -a team-a
ATESPACE   TEMPLATE NS        TEMPLATE   ID     STATUS             ATEOM POD   ATEOM IP   VERSION
team-a     ate-demo-counter   counter    test   STATUS_SUSPENDED   <none>                 5

Scope / non-goals

Deferred to later atespace increments (intentionally not in this PR): the Atespace object + CRUD
RPCs, DNS names, snapshot paths, template grants, the worker's own (system) atespace, and quota.

Tests pass
Appropriate documentation

Tim Hockin (thockin)

Currently an atespace is created merely by using it in an actor. I think we want to move to Atespaces being explicitly created and managed, right? I'm fine with that as a followup, or as more commits in here.

Tim Hockin (thockin) · 2026-06-22T21:55:16Z

+  string name = 1;
 }

 message GetActorRequest {


Should we just join name and atespace into a message and use it everywhere? It seems like there's never a context where we want one and not the other?

Tim Hockin (thockin) · 2026-06-22T22:00:19Z

+
+// Atespace is the tenant boundary an Actor is created into. Placeholder for now
+// (name only).
+message Atespace {


I assumed I would find CRUD APIs for Atespaces, but I don't see them?

Tim Hockin (thockin) · 2026-06-22T22:02:35Z

-	// Lists all known actors. Returns a page of actors and a next page token.
-	ListActors(ctx context.Context, pageSize int32, pageToken string) ([]*ateapipb.Actor, string, error)
+	// Lists actors in the given atespace (scoped scan). Returns a page of actors and a next page token.
+	ListActors(ctx context.Context, atespace string, pageSize int32, pageToken string) ([]*ateapipb.Actor, string, error)


Does "" mean "all atespaces" ?

Tim Hockin (thockin) · 2026-06-22T22:42:44Z

  Selector worker_selector = 4;
+
+  // The atespace to create the actor into.
+  string atespace = 5;


I appreciate the caution of no renumbering proto tags, but we should strive to make this as readable as possible, so I would put it next to ID

Agree, we are still in a phase, we can break API. Just put a comment in PR that this is a breaking change and all actors needs to be deleted.

In any case, all existing actors will not work, since they will miss namespace.

Tim Hockin (thockin) · 2026-06-22T23:33:02Z

 func init() {
 	createActorCmd.Flags().StringVarP(&templateFlag, "template", "t", "", "Template to derive the actor from in <namespace>/<name> format (required)")
 	_ = createActorCmd.MarkFlagRequired("template")
+	createActorCmd.Flags().StringVar(&atespaceFlag, "atespace", "", "Atespace (tenant) to create the actor in (required)")


Let's avoid the word "tenant"

Tim Hockin (thockin) · 2026-06-22T23:44:17Z


 		createReq := &ateapipb.CreateActorRequest{
 			ActorId:                actorID,
+			Atespace:               at.ObjectMeta.Namespace,


This is not obvious to me -- we should never assume k8s namespaces == atespaces -- they are not the same.

We are taking a snapshot, so we should probably use an atespace that is specifically put aside for this. If we move to CRUD of atespaces as a first-class resource, we should "reserve" any atespace whose name being with "ate-", so this can be something like "ate-golden"?

Tim Hockin (thockin) · 2026-06-22T23:52:51Z

 	dnsDomainParts := strings.Split("."+resources.ActorDNSSuffix+".", ".")
 	dnsDomainRef := strings.Join(dnsDomainParts, `\.`)
-	directives = append(directives, fmt.Sprintf(`  match "^%s%s$"`, resources.ActorIDRegexPattern, dnsDomainRef))
+	directives = append(directives, fmt.Sprintf(`  match "^%s\.%s%s$"`, resources.ActorIDRegexPattern, resources.ActorIDRegexPattern, dnsDomainRef))


This is fishy -- why do we need an explicit \. in one place but not the other?

Debugger says: the input is ".actors.resources.substrate.ate.dev." (leading and trailing dots). That makes dnsDomainParts have empty first and last entries. That means dnsDomainRef has leading and trailing \.. That makes this correct.

It's WEIRD but correct. I'll send a followup to document it

Tim Hockin (thockin) · 2026-06-22T23:58:22Z

+// "<actor_id>.<atespace>.actors.resources.substrate.ate.dev" (a trailing dot is
+// tolerated) into its atespace and actor id, validating both. It does not accept
+// a host:port; callers must strip the port first.
+func ParseActorDNSName(name string) (atespace, actorID string, err error) {


This seems like a perfect target for a small unit test?

Dmitry Berkovich (dberkov) · 2026-06-23T03:23:36Z

  Selector worker_selector = 4;
+
+  // The atespace to create the actor into.
+  string atespace = 5;


Agree, we are still in a phase, we can break API. Just put a comment in PR that this is a breaking change and all actors needs to be deleted.

In any case, all existing actors will not work, since they will miss namespace.

Dmitry Berkovich (dberkov) · 2026-06-23T03:27:57Z

+	if req.GetAtespace() == "" {
+		return status.Error(codes.InvalidArgument, "atespace is required")
+	}
+	if err := resources.ValidateAtespace(req.GetAtespace()); err != nil {


I am curious, why "resources.ValidateAtespace" does not validate for emptry string.
I see similar behavior is defined for resources.ValidateActorID too.
Might be we need to fix all the "resource validator" to check for empty string.

Julian Gutierrez Oschmann (@juli4n) - WDYT?

Dmitry Berkovich (dberkov) · 2026-06-23T03:33:30Z

 	if req.GetActorId() == "" {
 		return status.Error(codes.InvalidArgument, "id is required")
 	}
+	if req.GetAtespace() == "" {


have you considered to have full valudation? resources.ValidateNamespace.

It might worth to add similar full validation for actorID itself.

P.S - please make similar fix for all APIs, if you decide to accept the comment.

Haven Xia (HavenXia) added 3 commits June 18, 2026 22:25

Add atespace fields to ateapi proto

000d981

Scope actor store and handlers by atespace

b5f7474

Add --atespace flags to kubectl-ate

61d221b

Haven Xia (HavenXia) force-pushed the atespace-inc1 branch from e9eddfb to 1728f96 Compare June 19, 2026 05:44

Haven Xia (HavenXia) changed the title ~~Atespace inc1~~ The initial shape of atespace Jun 19, 2026

Haven Xia (HavenXia) changed the title ~~The initial shape of atespace~~ Atespace: per-tenant scoping for the actor lifecycle Jun 19, 2026

Haven Xia (HavenXia) added 3 commits June 19, 2026 12:39

List actors across all atespaces and show ATESPACE column

14c9e5a

kubectl-ate: -a shorthand for --atespace and TEMPLATE NS column

c52692f

Wire atespace through atecontroller golden actors and atenet routing

26d0074

Haven Xia (HavenXia) force-pushed the atespace-inc1 branch from 838b199 to 26d0074 Compare June 19, 2026 19:39

Haven Xia (HavenXia) marked this pull request as ready for review June 19, 2026 19:41

Tim Hockin (thockin) reviewed Jun 22, 2026

View reviewed changes

Dmitry Berkovich (dberkov) reviewed Jun 23, 2026

View reviewed changes

Conversation

Haven Xia (HavenXia) commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR changes:

Examples

Creation

Get by atespaces

Resume & Suspend

Uh oh!

Tim Hockin (thockin) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Haven Xia (HavenXia) commented Jun 19, 2026 •

edited

Loading